Synaptic weights that correlate with presynaptic selectivity increase decoding performance

The activity of neurons in the visual cortex is often characterized by tuning curves, which are thought to be shaped by Hebbian plasticity during development and sensory experience. This leads to the prediction that neural circuits should be organized such that neurons with similar functional preference are connected with stronger weights. In support of this idea, previous experimental and theoretical work have provided evidence for a model of the visual cortex characterized by such functional subnetworks. A recent experimental study, however, have found that the postsynaptic preferred stimulus was defined by the total number of spines activated by a given stimulus and independent of their individual strength. While this result might seem to contradict previous literature, there are many factors that define how a given synaptic input influences postsynaptic selectivity. Here, we designed a computational model in which postsynaptic functional preference is defined by the number of inputs activated by a given stimulus. Using a plasticity rule where synaptic weights tend to correlate with presynaptic selectivity, and is independent of functional-similarity between pre- and postsynaptic activity, we find that this model can be used to decode presented stimuli in a manner that is comparable to maximum likelihood inference.


Introduction
Neurons in the visual cortex are selectively driven by specific features of sensory stimuli. This selective neural activity is proposed to be shaped by Hebbian plasticity [1] during developmental stages and sensory experience. Hebbian plasticity can be described as the strengthening of synaptic weights in a manner dependent on the correlation of the pre-and postsynaptic neurons [2,3]. Thus, in the development of visual cortical circuits, Hebbian plasticity is thought to lead to a functional distribution in synaptic weights: weights are larger between pre-and postsynaptic neurons with similar functional preferences, i.e. similar preferred orientation [4] and only a few synaptic inputs are needed to define postsynaptic sensory feature selectivity [5]. In support of this framework, excitatory pyramidal neurons in layers 2/3 of mice primary visual cortex are shown to form functional subnetworks; the synaptic weights between neurons and the probability of connectivity between neurons reflect the similarity in tuning to visual features, such as for example orientation preference [6][7][8][9][10].
A recent study [11], however, has shown different results in the visual cortex of ferrets. Instead of a functionally-defined weight distribution, [11] found that preference for orientation stimuli is derived from the total number of excitatory synaptic inputs activated by a given stimulus. That is, strong and weak synapses were recruited for all visual stimuli presented. While this result appears to contradict the previous literature, there are many factors which impact how a given synaptic input might influence activity at the somatic output and postsynaptic selectivity. Some of these factors include synapse weight and number, reliability [12], location within the dendritic tree [13], co-activity with their neighbors [14], and presence of dendritic inhibition [15]. In fact, given that pairs of cortical neurons tend to be connected through multiple synaptic contacts [16][17][18][19], an overall stronger weight between pairs of neurons with correlated activity could be achieved through many contacts with a mixture of weights. In this case, it is possible that individual synaptic weights might depend on a learning rule that is independent of the postsynaptic neuron's activity. Such a learning rule would allow flexibility in weighting and encode something other than covariance between presynaptic and postsynaptic activity. For example, [11] also showed that anatomical correlates of synaptic strength correlate with spine selectivity, i.e. the sharpness of the tuning curve or how selectively a spine responds to specific orientations. This observation leads to an interesting set of questions: (1) Can a presynaptic learning rule generate strong, selective synapses? (2) Is a presynaptic learning rule sufficient to generate selective postsynaptic neurons? (3) Does a presynaptic learning rule lead to efficient neuronal population decoding?
To address these questions, we model a feedforward circuit with a plasticity rule leading to synaptic weights that are correlated with presynaptic selectivity (termed 'variance rule'), independent of the difference between pre-and postsynaptic preferred orientation (ΔPO). Using this model, we explore how synaptic weights based on presynaptic selectivity, rather than functional-similarity between presynaptic and postsynaptic activity, might be achieved by cortical neurons and what impacts this distribution has on neuron decoding. We show that this plasticity rule can lead to postsynaptic neurons selectively tuned to specific stimuli. Furthermore, we show that a decoder which uses the weights emerging from the plasticity rule performs comparatively to a decoder derived based on maximum likelihood inference. Overall, our results suggest a decoding model where the somatic preference is defined primarily by the number of activated spines given a certain stimulus, while the strength of individual spines correlates with the tuning selectivity of the corresponding inputs and does not entirely depend on the tuning similarity with the postsynaptic neuron.

Results
Here we study the functional implications of a plasticity rule based on presynaptic variance, rather than covariance between presynaptic and postsynaptic activity (i.e. classic Hebbian), in a feedforward circuit representing neurons in the primary visual cortex. Our circuit is composed of one postsynaptic neuron, modeled as a point neuron (see Methods for details), which receives input from N presynaptic neurons through plastic weights w i , and from an untuned inhibitory source (Fig 1A). Presynaptic neurons are orientation selective and their activity is modulated by tuning curves (Fig 1B). In order to model a diverse population (i.e. differences in orientation preference and selectivity), the tuning curve of each presynaptic neuron i has its own preferred orientation (PO, y P i ) and width (κ i ) ( Fig 1B).

A plasticity rule based on presynaptic variance results in weights that correlate with presynaptic selectivity
In order to build a model where synaptic weights correlate with presynaptic selectivity, we first study how synaptic plasticity might lead to this correlation. Based on previous studies showing that some forms of LTP can be induced without the need of postsynaptic depolarization [20][21][22][23], we propose a plasticity rule in which potentiation is based on presynaptic activity only. More specifically, a rule in which potentiation is based on presynaptic variance (variance rule).
In this way, changes in synaptic weight, Δw, can be summarized as where r pre is the presynaptic activity, μ is a constant representing the mean presynaptic activity, which is the same for all presynaptic neurons, η 1 is the learning rate and η 0 is the weight decay rate. We then stimulate the presynaptic neurons by showing all of them a stimulus of orientation θ. Every 200 ms, a new stimulus θ is randomly chosen from a uniform distribution between À p 2 ; p 2 � � and shown to all presynaptic neurons. We then observe the evolution of the synaptic weights ( Fig 1C).
Once converged, the synaptic weights have a skewed distribution (Fig 1D), and are essentially a function of the width of the presynaptic tuning curve (Fig 1E). Since selectivity of a presynaptic neuron is given by the width of its tuning curve κ (Fig 1F), the converged weights are also correlated with presynaptic selectivity (Fig 1G). The section "Plasticity weights" in Methods provides the equation for the equilibrium weights from a presynaptic neuron i, showing that it is indeed a function of its selectivity κ i . Accordingly, initializing the weights from random values drawn from a uniform distribution leads to similar results (S7 Fig). Thus, we show that a feedforward model with a plasticity rule based on the variance of presynaptic activity, leads to synaptic weights that are correlated with presynaptic selectivity.

Postsynaptic neurons are orientation selective and synaptic weights are uncorrelated with the difference in preferred orientation between pre-and postsynaptic neuron (ΔPO)
In the previous simulations, we find that, similar to the presynaptic neurons, the postsynaptic neuron is also orientation tuned (Fig 2B). Extracting the relevant parameters from the postsynaptic tuning curve across multiple simulation runs shows that individual postsynaptic neurons respond preferentially to orientations spread across the full range of stimuli À p 2 ; p 2 � � and with selectivity values that cover almost the entire range [0, 1] (Fig 2C). Note that the value of selectivity depends on the mean activity level [24], which in our simulations Synaptic plasticity rules which are based on the covariance between pre-and postsynaptic activity generate circuits in which the strength of synaptic weights are anti-correlated with ΔPO (S2 Fig). That is, if the pre-and postsynaptic neurons have similar tuning, they typically are connected with a strong weight. But this is not necessarily the case for a rule based on the variance of presynaptic activity only. Therefore, we next test whether the synaptic weights are anti-correlated with ΔPO. Similar to experimental data [11], we find that the synaptic weights are not correlated with ΔPO and depend mostly on presynaptic selectivity (Fig 2A and 2D). Even though there is no clear relationship between the synaptic weights and ΔPO, the activity of the postsynaptic neuron is still more correlated with the activity of presynaptic neurons that have similar POs (Fig 2E).

Postsynaptic neurons inherit preferred orientation from the number of presynaptic inputs with similar preference
If it is not true that synaptic weights are stronger when pre-and postsynaptic neurons have similar POs, what is then defining the postsynaptic orientation preference? While some previous work has proposed that somatic functional preference is derived from the functional preference of a few stronger presynaptic inputs [9,10], others have proposed that it is actually defined by the number of active spines with a given PO [11]. Since in our model the synaptic weights are not correlated with ΔPO (Fig 2D), we expect that somatic preference will be defined by the number of presynaptic inputs with similar preferences.
In order to test this, we wanted to know whether all orientations within the interval À p 2 ; p 2 � � were equally represented within presynaptic populations or if there was any bias towards the PO of the postsynaptic neuron. Even though the PO of the presynaptic inputs were independently drawn from a uniform distribution (Fig 3A), we find that there are indeed slightly more presynaptic inputs which are co-tuned than inputs which are not co-tuned with the postsynaptic neuron ( Fig 3B). To further explore this result, we calculate the total input current to the postsynaptic neuron when different orientations are being shown to the presynaptic population ( Fig 3C). We then split the total input current into two values: the total number of active inputs and the mean synaptic weight of those active inputs. Inputs are considered to be active when their firing rate is above a certain threshold during the presentation of a stimulus (see Methods for details). We find that, similar to the total input current (Fig 3C), the number of active inputs is also modulated by the stimulus, such that a larger number is active when the PO is shown (Fig 3D). Importantly, we did not observe any difference in the main results by varying the value of the threshold (S10 and S11 Figs). We also found that this bias in the total input current is still present when varying the number of presynaptic neurons (N = 10, 100, 1000) and the bias is enhanced by decreasing the total number of presynaptic inputs (S4-S6 Figs). We also found that the bias is amplified by a bias in the preferred orientation of the presynaptic neurons, simulated by drawing presynaptic PO from a normal distribution (S9 Fig).
Unexpectedly, however, we find that the mean weight of the active inputs is also modulated by the difference between the shown stimulus and the postsynaptic preferred stimulus (Fig 3E). This is different to what was observed in experiments [11] and counter-intuitive given that the plasticity rule used here is based on presynaptic activity only. However, even though correlated activity between pre-and postsynaptic neurons does not lead directly to stronger weights, the postsynaptic PO is given by a sum of the presynaptic POs weighted by their selectivity and by their input weights (S3 Fig). Therefore, the postsynaptic PO will be biased by presynaptic neurons with stronger weights, which are also the ones with higher selectivity (S3 Fig).

A decoder based on the variance rule performs comparatively to a decoder based on maximum likelihood
Hebbian plasticity has been previously shown to enhance orientation selectivity [25], in accordance with experimental studies showing an increase in orientation selectivity after visual experience [8,26]. Since Hebbian plasticity shapes circuits in which synaptic weights are anticorrelated with ΔPO (S2 Fig), the question emerges of whether there is any computational advantage of synaptic weights that correlate with presynaptic selectivity instead. Therefore we next study a feedforward circuit where the synaptic weights are correlated with presynaptic selectivity in the context of decoding stimuli.
We assume there is a population of input neurons with different tuning curves ( Fig 4A). The same orientation θ is shown to all input neurons and the firing rate of presynaptic neuron i is given by the shown stimulus and their respective tuning curve, defined by their PO y P i and width κ i . A decoder receives inputs from all neurons in this population and estimates the shown orientation based on their tuning curves and firing rates [27]. We then derive a decoder, which is based on maximum likelihood inference (ML decoder, see Methods for details) [28]. Assuming the input tuning curves to be modeled as von Mises functions, the ML decoder estimates the shown orientation with (Fig 4B): where p i is the firing rate of neuron i (r i with added Poisson noise, see Methods for details), κ i is a parameter of the von Mises tuning curve defining its width, y P i is the preferred stimulus of input neuron i andŷ is the orientation estimated by the ML decoder.
Similar to Bayesian cue integration, in which inputs are integrated by being multiplied by their uncertainty [29][30][31], this decoder is such that the firing rate p i of input neuron i is being multiplied by the width of its tuning curve κ i , which defines its selectivity (Eq 2). This multiplication p i κ i could be easily implemented by synaptic weights which are correlated with presynaptic selectivity (Fig 1). Therefore, we next compare the performance of the ML decoder with that of a decoder that uses the weights obtained from the variance rule (Fig 1). In order to do this, we assume that the decoder multiplies the firing rate of each presynaptic neuron i by a weight w i in order to estimate the shown orientation θ: where w i are the weights used by the decoder and w i = κ i in the case of the ML decoder.

PLOS COMPUTATIONAL BIOLOGY
We then compare the performance of the decoder that uses the weights obtained from the variance rule (Variance Rule) to other decoders (Fig 4C), namely: (i) to a decoder where w i = κ i (ML); (ii) to a decoder that uses the same weight for all input neurons w i = 1 (Uniform); (iii) to a decoder that uses the same weights as the Variance Rule decoder but shuffled, and therefore uncorrelated to presynaptic selectivity (Shuffled); (iv) to a decoder that uses the weights obtained from a simulation using a covariance plasticity rule (Covariance, see). We find that the presynaptic variance decoder performs comparatively to the ML decoder and better than the others (Fig 4D). We did not find any qualitative difference in the main results by adding noise to the input coming from the presynaptic neurons (S12 and S13 Figs).
In conclusion, we demonstrate how a plasticity rule based on presynaptic variance can lead to the formation of feedforward circuits where the postsynaptic neurons are selective for specific stimuli. Under this framework, we find that postsynaptic neurons can act as decoders whose performance rivals that of maximum likelihood inference. These results suggest a decoding model where individual inputs received by the postsynaptic neuron are weighted according to presynaptic selectivity rather than functional-similarity between pre-and postsynaptic activity.

Discussion
In this work, we simulated a feedforward network in which weights were changing according to a plasticity rule based on the variance of presynaptic activity. The presynaptic population was composed of neurons with diverse preferred orientation (PO) and selectivity. We showed that, by stimulating the input population with different orientations, synaptic weights converged to values that were a function of presynaptic selectivity. We then showed that a decoder which used those weights to decode a stimulus had a performance comparable to a decoder based on maximum likelihood inference.
We focused on the computational implications of synaptic weights that correlate with presynaptic selectivity, but did not explore the functional advantages of somatic preference being defined by the number of spines rather than by their strength [11]. This could be interesting to explore in view of recent experimental evidence showing that learning is associated with the formation of new spines [32][33][34][35] and previous theoretical work suggesting that having multiple synaptic contacts between the same pair of neurons, instead of having a single strong one, could lead to more robust circuits with stable memories [36][37][38]. A possible extension to our model could be to include structural plasticity that causes multiple contacts between pre-and postsynaptic neurons to form. In addition, this model could be extended to include both a covariance rule and a presynaptic variance rule for individual neurons, as including only the latter results in fixed synaptic weights for any given presynaptic neuron. Finally, in the future, it will be interesting to study how a presynaptic variance rule can be applied to a recurrent network receiving inputs from multiple sources (e.g. feed-forward, long-range) and whether such models could better describe the development of visual cortical circuits.
Different structural plasticity rules have been proposed to describe activity-dependent spine turnover [39]. A structural plasticity rule forming stronger connectivity between neurons with similar orientation preference [40], could lead to a circuit where more synaptic contacts are formed between the same pair of pre-and postsynaptic neurons when their activity is correlated ( Fig 2E). As a consequence, there could be an even stronger bias in the number of presynaptic inputs that are co-tuned with the soma (larger number of inputs with |ΔPO| = 0 in Fig  3B). Another possible consequence could be a stronger overall synaptic weight due to an increased number of synaptic contacts between co-tuned pre-and postsynaptic neurons. Using a modified simulation of our model to include multiple contacts, such a scenario appears very plausible and could reconcile different plasticity rules (see S14 Fig). This could be interesting when comparing experimental data derived from measuring synaptic weights as anatomical features of spines versus measuring synaptic weights as amplitudes of postsynaptic potentials recorded from the soma. On the other hand, using a structural plasticity rule in which weaker weights are more likely to be deleted than stronger ones [37,41] could lead to different functional circuits.
Previous work has shown that the diversity of synaptic weights found in the visual cortex of ferrets could be explained by a model where a decoder reads information from a population of diverse PO and selectivity [42]. Here, we propose that the synaptic weights could reflect presynaptic selectivity. Such diversity in synaptic weights, therefore, would not be necessary if all presynaptic inputs had the same selectivity. Why then would there be a population of neurons with diverse selectivity in the visual cortex? One possibility is that decoding from a population of input neurons with a range of selectivity provides better discrimination capabilities for natural images [43]. Another option is that selectivity could somehow encode presynaptic uncertainty, but how exactly uncertainty is encoded in neural representation is still unclear [44].
According to the rule presented in this paper, synaptic weights would fluctuate as the selectivity changes. Assuming that selectivity would reflect uncertainty, synaptic weight fluctuations would also reflect presynaptic uncertainty. In this case, we refer to uncertainty as the amount of information a particular synapse is able to convey to the postsynaptic target, which would change either from changes in the presynaptic neuron's activity or noise within incoming sensory information. This is similar to what has been recently proposed in a theoretical paper [45], and in accordance with experimental data showing that synaptic weights fluctuate over time [46,47]. In contrast to our model, however, experimental studies have shown that the weights can also fluctuate in an activity independent manner. Secondly, we might expect changes in synaptic weights to cause changes in the postsynaptic neuron's activity. The postsynaptic PO would therefore change over time, reflecting changes in the reliability of presynaptic inputs. Interestingly, this possibility might correspond to recent experimental findings of 'representational drift', where neurons of the visual cortex and other brain regions are reportedly to exhibit fluctuations in their tuning properties over time [48].
In conclusion, our results suggest a model of the visual cortex in which postsynaptic preferred orientation is defined by the number of spines with a given preferred orientation, while synaptic strength is used as a weight reflecting presynaptic selectivity.

Methods
The code for simulations and figures is available at https://github.com/juliavg/decoding.

Neuron model
Output neuron. The output neuron is modeled as a rate based neuron. Its firing rate y at time t is given by the equation: " where τ y = 1 ms is the rate time constant, α = 0.1 nA −1 is the slope of the neuron's transfer function, w ref = 16 nA is a reference weight for the N excitatory inputs, w i and r i are respectively the weight and firing rate of input neuron i, w I = −1700 pA and r I = 100 Hz are respectively the weight and firing rate of the inhibitory source. The inhibitory source could be understood as a single neuron firing at 100 Hz or multiple neurons firing at a lower rate and adding up to the same value. The sign [] + indicates a rectification that sets all negative values to 0. Input neurons. The input neurons are modeled as rate based neurons with von Mises tuning curves, which is a "bump" like curve with circular boundary conditions. The firing rate of input neuron i at time t is given by: where r ref = 125 Hz is a reference firing rate, θ is the stimulus being shown to neuron i at time t, κ i is a parameter defining the width of the tuning curve from neuron i, y P i is the preferred orientation of neuron i and I 0 is the modified Bessel function of order 0.

Plasticity models
Variance (Figs 1-3). The weights from the excitatory inputs are plastic, and evolve according to: where η 1 = 0.1 and η 0 = 0.03 are learning rates and m ¼ 1 2p is a constant representing the mean activity of the presynaptic neurons. The same constant m ¼ 1 2p is used for all presynaptic neurons.
Covariance (S2 Fig). For the simulations with the covariance rule, the excitatory weights evolve according to: where γ = 0.24 is a constant and the remaining parameters are the same as with the rule based on presynaptic variance.

Plasticity simulation (Figs 1-3)
One output neuron receives input from N = 50 neurons. Each input neuron i has a tuning curve defined by its width κ i , which is drawn randomly and independently for each neuron from a uniform distribution ]0, 1], and by its preferred orientation y P i , which is drawn randomly and independently for each neuron from a uniform distribution À p 2 ; p 2 � � . The firing rate of the input neurons are initially set to r i = 20 Hz for a warm up period of 200 s, after which the stimulation protocol starts. During the stimulation protocol, a new orientation θ is chosen randomly from a uniform distribution À p 2 ; p 2 � � every T = 200 ms. The same orientation θ is shown to all input neurons and the whole stimulation protocol consists of 1 000 stimuli. We run 100 independent simulation runs, and shown in the figures is the pooled data from all of them.

Orientation selectivity
Preferred orientation and orientation selectivity of neuron i are calculated from the circular mean of neuronal response R i : where c i (θ) is the calculated tuning curve of neuron i. We calculate c i (θ) as the mean response across stimuli using 20 bins on the interval À p 2 ; p 2 � � and using data from the last 500 stimuli in the simulation. Preferred orientation y P i and the selectivity of neuron i are then calculated as the angle and the length of the resultant R i , respectively.

Plasticity weights
The equilibrium weights for the plasticity rule based on presynaptic variance can be calculated by setting the left hand side of Eq 6 to zero: Assuming that the weights reach the steady state W within the interval T (how long each orientation is shown), we substitute θ(t) by the random variable Θ and take expected values: Finding E � r i ðYÞ ; and substituting it in Eq 10 gives the equilibrium weight from presynaptic neuron i: Active input analysis (Fig 3) For the active input analysis, we compare the mean activity of each input neuron during the presentation of a single stimulus to a threshold b ¼ r ref 2p and consider it to be active (not active) when the activity is above (below) β.

Maximum likelihood decoder
From [28], the estimated stimulus (ŷ) of a decoder based on maximum likelihood inference (ML decoder) can be determined by: where p i is the firing rate of neuron i, r i (θ) is the tuning curve of neuron i, and the prime denotes derivative. Assuming the tuning curves of the simulation input neurons (Eq 5): then the estimated stimulusŷ can be determined by: Solving forŷ gives:ŷ See also [49] for a similar derivation.
Decoder simulation (Fig 4) A decoder infers the stimulus θ shown to a population of 50 input neurons. The activity of input neuron i in response to stimulus θ (p i ) is given as a sample randomly drawn from the Poisson distribution: where f (k, r i (θ)) describes the probability of neuron i firing with rate k in response to stimulus θ, and r i (θ) is the tuning curve of neuron i (Eq 5).
The decoder estimates the shown stimulus according to: where p i is the firing rate of neuron i (obtained from sampling from Eq 16), w i are the weights used by the decoder and y P i is the preferred orientation of neuron i. The tuning curves and parameters of input neurons are taken from the 100 independent plasticity simulations.
We compare the performance of 5 different decoders, which differ based on the weights w i they use: (i) variance: the weights used are obtained from the corresponding plasticity simulation; (ii) ML: w i = κ i ; (iii) uniform: w i = 1; (iv) shuffled: the weights are the same as the variance, but shuffled; (v) covariance: the weights used are obtained from the plasticity simulations using the covariance plasticity rule.
For each independent plasticity simulation, we evaluate the decoder by showing 20 orientations equally spaced in the interval À p 2 ; p 2 � � . Each orientation is shown repeatedly for 100 trials, and we calculate the bias b est (θ), variance s 2 est ðyÞ and error e est (θ) using: where hi indicates the average across trails. For each independent plasticity simulation, we average the bias b est (θ), variance s 2 est ðyÞ and error e est (θ) across stimuli θ to obtain a single value per simulation. We then purposely manipulate the connectivity of two presynaptic neurons N 1 and N 2 . For the first presynaptic neuron N 1 , we choose its preferred orientation to be the same as the postsynaptic neuron (which is given by the mean 0 of the distribution of presynaptic preferred orientations) and we assume it has a low selectivity by setting its κ = 0.5. Assuming the number of synaptic contacts between the pre-and the postsynaptic neuron would reflect the correlation between them, we create 5 contacts between this presynaptic neuron N 1 and the postsynaptic neuron. The individual strengths of all these 5 contacts are subject to the plasticity rule based on presynaptic variance. For the second presynaptic neuron N 2 , we choose its preferred orientation to be orthogonal to the postsynaptic neuron ( p 2 ) and assume it would have high selectivity by setting its κ = 1. Since N 2 has low correlation with the postsynaptic neuron, we create only a single synaptic contact between them. All the other presynaptic neurons have only a single synaptic contact to the postsynaptic neuron, their preferred orientations are drawn randomly from a normal distribution with mean 0 and standard deviation p 8 , and their selectivity κ are randomly drawn from a uniform distribution as in the main simulations of the manuscript. We then run the simulation in the same way as the main simulation in the manuscript. As a result, while the weight of individual synaptic contacts is smaller between N 1 and the postsynaptic neuron than between N 2 and postsynaptic neuron, the overall pre-post synaptic weight (that considers both number of contacts and strength of individual contact) is larger between N 1 and the postsynaptic neuron than between N 2 and the postsynaptic neuron.